Barriers to and facilitators of interventions to counter publication bias: thematic analysis of scholarly articles and stakeholder interviews.

BACKGROUND: When the nature and direction of research results affect their chances of publication, a distortion of the evidence base - termed publication bias - results. Despite considerable recent efforts to implement measures to reduce the non-publication of trials, publication bias is still a major problem in medical research. The objective of our study was to identify barriers to and facilitators of interventions to prevent or reduce publication bias. METHODS: We systematically reviewed the scholarly literature and extracted data from articles. Further, we performed semi-structured interviews with stakeholders. We performed an inductive thematic analysis to identify barriers to and facilitators of interventions to counter publication bias. RESULTS: The systematic review identified 39 articles. Thirty-four of 89 invited interview partners agreed to be interviewed. We clustered interventions into four categories: prospective trial registration, incentives for reporting in peer-reviewed journals or research reports, public availability of individual patient-level data, and peer-review/editorial processes. Barriers we identified included economic and personal interests, lack of financial resources for a global comprehensive trial registry, and different legal systems. Facilitators identified included: raising awareness of the effects of publication bias, providing incentives to make data publically available, and implementing laws to enforce prospective registration and reporting of clinical trial results. CONCLUSIONS: Publication bias is a complex problem that reflects the complex system in which it occurs. The cooperation amongst stakeholders to increase public awareness of the problem, better tailoring of incentives to publish, and ultimately legislative regulations have the greatest potential for reducing publication bias.

Meta-analysis: a unifying meta-likelihood approach framing unobserved heterogeneity, study covariates, publication bias, and study quality

OBJECTIVES: This contribution provides a unifying concept for meta-analysis integrating the handling of unobserved heterogeneity, study covariates, publication bias and study quality. It is important to consider these issues simultaneously to avoid the occurrence of artifacts, and a method for doing so is suggested here. METHODS: The approach is based upon the meta-likelihood in combination with a general linear nonparametric mixed model, which lays the ground for all inferential conclusions suggested here. RESULTS: The concept is illustrated at hand of a meta-analysis investigating the relationship of hormone replacement therapy and breast cancer. The phenomenon of interest has been investigated in many studies for a considerable time and different results were reported. In 1992 a meta-analysis by Sillero-Arenas et al. concluded a small, but significant overall effect of 1.06 on the relative risk scale. Using the meta-likelihood approach it is demonstrated here that this meta-analysis is due to considerable unobserved heterogeneity. Furthermore, it is shown that new methods are available to model this heterogeneity successfully. It is argued further to include available study covariates to explain this heterogeneity in the meta-analysis at hand. CONCLUSIONS: The topic of HRT and breast cancer has again very recently become an issue of public debate, when results of a large trial investigating the health effects of hormone replacement therapy were published indicating an increased risk for breast cancer (risk ratio of 1.26). Using an adequate regression model in the previously published meta-analysis an adjusted estimate of effect of 1.14 can be given which is considerably higher than the one published in the meta-analysis of Sillero-Arenas et al. In summary, it is hoped that the method suggested here contributes further to a good meta-analytic practice in public health and clinical disciplines.

Publication bias in the economic freedom and economic growth literature

The impact of institutions on economic performance has attracted significant attention from researchers, as well as from policy reformers. A rapidly growing area in this literature is the impact of economic freedom on economic growth. The aim of this paper was to explore publication bias in this literature by means of traditional funnel plots, meta‐significance testing, as well as by bootstrapping these meta‐significance tests. When all the available estimates are combined and averaged, there seems to be evidence of a genuine and positive economic freedom – economic growth effect. However, it is also shown that the economic freedom – economic growth literature is tainted strongly with publication bias. The existence of publication bias makes it difficult to identify the magnitude of the genuine effect of economic freedom on economic growth. The paper explores the differences between aggregate and disaggregate measures of economic freedom and shows that selection effects are stronger when aggregate measures are used.

Publication bias in union-productivity research?

This paper develops and applies several meta-analytic techniques to investigate the presence of publication bias in industrial relations research, specifically in the union-productivity effects literature. Publication bias arises when statistically insignificant results are suppressed or when results satisfying prior expectations are given preference. Like most fields, research in industrial relations is vulnerable to publication bias. Unlike other fields such as economics, there is no evidence of publication bias in the union-productivity literature, as a whole. However, there are pockets of publication selection, as well as negative autoregression, confirming the controversial nature of this area of research. Meta-regression analysis reveals evidence of publication bias (or selection) among US studies.

Publication bias in industrial relations research

Publication bias arises when statistically non-significant results are suppressed or when only results satisfying prior expectations are published. Like most fields, research in industrial relations is vulnerable to publication bias. In this paper qualitative and quantitative techniques are used in order to detect publication bias in the union-productivity effects literature. We find no evidence of publication bias in this literature, although there does appear to be autoregression in the published results.

Publication Bias in Five Dental Implant Journals: An Observation from 2005 to 2009

Purpose: This study evaluated possible publication bias and its related factors in implant-related research over time. Materials and Methods: Articles published in Clinical Implant Dentistry and Related Research, Clinical Oral Implants Research, Implant Dentistry, Journal of Oral Implantology, and The International Journal of Oral & Maxillofacial Implants between 2005 and 2009 were reviewed. Nonoriginal articles were excluded. For each article included, study outcome, extramural funding source, type of study, and geographic origin were recorded. Descriptive and analytic statistics (alpha = .05), including the chi-square test and logistic regression analysis, were performed where appropriate. Results: From a total of 2,085 articles, 1,503 met the inclusion criteria. of the articles analyzed, 1,226 (81.6%), 160 (10.6%), and 117 (7.8%) articles reported positive, negative, and neutral outcomes, respectively. In vitro studies, studies from Asia, and funded animal studies were more likely to report positive outcomes compared to others (P = .02, P < .0001, and P = .009, respectively). Industry-funded studies represented the lowest frequency of positive outcomes versus studies funded by other sources. Conclusions: There were a high number of implant-related studies reporting positive outcomes in the five selected journals. Some selected factors were associated with positive outcome bias. In general, funding was not associated with a positive outcome, except for animal studies. Industry-supported research did not show any association with the publication of positive outcomes. INT J ORAL MAXILLOFAC IMPLANTS 2011;26:1024-1032

Assessment of publication bias in dental specialty journals

OBJECTIVE: The purpose of this study was to investigate the presence of publication bias (acceptance of articles indicating statistically significant results). METHODS: The journals possessing the highest impact factor (2008 data) in each dental specialty were included in the study. The content of the 6 most recent issues of each journal was hand searched and research articles were classified into 4 type categories: cross-sectional, case-control, cohort, and interventional (nonrandomized clinical trials and randomized controlled trials). In total, 396 articles were included in the analysis. Descriptive statistics and univariate and multivariate logistic regression was used to examine the association between article-reported statistical significance (dependent variable) and journal impact factor and article study type subject area (independent variables). RESULTS: A statistically significant acceptance rate of positive result was found, ranging from 75% to 90%, whereas the value of impact factor was not related to publication bias among leading dental journals. Compared with other research designs, clinical intervention studies (randomized or nonrandomized) presented the highest percentage of nonsignificant findings (20%); RCTs represented 6% of the examined investigations. CONCLUSIONS: Compared with the Journal of Clinical Periodontology, all other subspecialty journals, except the Journal of Oral and Maxillofacial Surgery, showed significantly decreased odds of publishing an RCT, which ranged from 60% to 93% (P < .05).

Systematic review of the empirical evidence of study publication bias and outcome reporting bias

BACKGROUND: The increased use of meta-analysis in systematic reviews of healthcare interventions has highlighted several types of bias that can arise during the completion of a randomised controlled trial. Study publication bias has been recognised as a potential threat to the validity of meta-analysis and can make the readily available evidence unreliable for decision making. Until recently, outcome reporting bias has received less attention. METHODOLOGY/PRINCIPAL FINDINGS: We review and summarise the evidence from a series of cohort studies that have assessed study publication bias and outcome reporting bias in randomised controlled trials. Sixteen studies were eligible of which only two followed the cohort all the way through from protocol approval to information regarding publication of outcomes. Eleven of the studies investigated study publication bias and five investigated outcome reporting bias. Three studies have found that statistically significant outcomes had a higher odds of being fully reported compared to non-significant outcomes (range of odds ratios: 2.2 to 4.7). In comparing trial publications to protocols, we found that 40-62% of studies had at least one primary outcome that was changed, introduced, or omitted. We decided not to undertake meta-analysis due to the differences between studies. CONCLUSIONS: Recent work provides direct empirical evidence for the existence of study publication bias and outcome reporting bias. There is strong evidence of an association between significant results and publication; studies that report positive or significant results are more likely to be published and outcomes that are statistically significant have higher odds of being fully reported. Publications have been found to be inconsistent with their protocols. Researchers need to be aware of the problems of both types of bias and efforts should be concentrated on improving the reporting of trials.

Publication Bias

According to the Declaration of Helsinki, as well as the Statement on Public Disclosure of Clinical Trial Results of the World Health Organization, every researcher has the ethical obligation to publish research results on all trials with human participants in a complete and accurate way within 12 months after the end of the trial.1,2 Nevertheless, for several reasons, not all research results are published in an accurate way in case they are released at all. This phenomenon of publication bias may not only create a false impression on the reliability of clinical research business, but it may also affect the evidence of clinical conclusions about the best treatments, which are mostly based on published data and results.

Full publication of results initially presented in abstracts

BACKGROUND: Abstracts of presentations at scientific meetings are usually available only in conference proceedings. If subsequent full publication of abstract results is based on the magnitude or direction of study results, publication bias may result. Publication bias, in turn, creates problems for those conducting systematic reviews or relying on the published literature for evidence. OBJECTIVES: To determine the rate at which abstract results are subsequently published in full, and the time between meeting presentation and full publication. To assess the association between study characteristics and full publication. SEARCH STRATEGY: We searched MEDLINE, EMBASE, The Cochrane Library, Science Citation Index, reference lists, and author files. Date of most recent search: June 2003. SELECTION CRITERIA: We included all reports that examined the subsequent full publication rate of biomedical results initially presented as abstracts or in summary form. Follow-up of abstracts had to be at least two years. DATA COLLECTION AND ANALYSIS: Two reviewers extracted data. We calculated the weighted mean full publication rate and time to full publication. Dichotomous variables were analyzed using relative risk and random effects models. We assessed time to publication using Kaplan-Meier survival analyses. MAIN RESULTS: Combining data from 79 reports (29,729 abstracts) resulted in a weighted mean full publication rate of 44.5% (95% confidence interval (CI) 43.9 to 45.1). Survival analyses resulted in an estimated publication rate at 9 years of 52.6% for all studies, 63.1% for randomized or controlled clinical trials, and 49.3% for other types of study designs.'Positive' results defined as any 'significant' result showed an association with full publication (RR = 1.30; CI 1.14 to 1.47), as did 'positive' results defined as a result favoring the experimental treatment (RR =1.17; CI 1.02 to 1.35), and 'positive' results emanating from randomized or controlled clinical trials (RR = 1.18, CI 1.07 to 1.30).Other factors associated with full publication include oral presentation (RR = 1.28; CI 1.09 to 1.49); acceptance for meeting presentation (RR = 1.78; CI 1.50 to 2.12); randomized trial study design (RR = 1.24; CI 1.14 to 1.36); and basic research (RR = 0.79; CI 0.70 to 0.89). Higher quality of abstracts describing randomized or controlled clinical trials was also associated with full publication (RR = 1.30, CI 1.00 to 1.71). AUTHORS' CONCLUSIONS: Only 63% of results from abstracts describing randomized or controlled clinical trials are published in full. 'Positive' results were more frequently published than not 'positive' results.

Full-text publication of abstracts presented at European Orthodontic Society congresses.

INTRODUCTION Empirical evidence has indicated that only a subsample of studies conducted reach full-text publication and this phenomenon has become known as publication bias. A form of publication bias is the selectively delayed full publication of conference abstracts. The objective of this article was to examine the publication status of oral abstracts and poster-presentation abstracts, included in the scientific program of the 82nd and 83rd European Orthodontic Society (EOS) congresses, held in 2006 and 2007, and to identify factors associated with full-length publication. METHODS A systematic search of PubMed and Google Scholar databases was performed in April 2013 using author names and keywords from the abstract title to locate abstract and full-article publications. Information regarding mode of presentation, type of affiliation, geographical origin, statistical results, and publication details were collected and analyzed using univariable and multivariable logistic regression. RESULTS Approximately 51 per cent of the EOS 2006 and 55 per cent of the EOS 2007 abstracts appeared in print more than 5 years post congress. A mean period of 1.32 years elapsed between conference and publication date. Mode of presentation (oral or poster), use of statistical analysis, and research subject area were significant predictors for publication success. LIMITATIONS Inherent discrepancies of abstract reporting, mainly related to presentation of preliminary results and incomplete description of methods, may be considered in analogous studies. CONCLUSIONS On average 52.2 per cent of the abstracts presented at the two EOS conferences reached full publication. Abstracts presented orally, including statistical analysis, were more likely to get published.

Difference-based meta-analytic procedures for between-participant and/or within-participant designs : a tutorial review for sports and exercise scientists

The aim of this paper is to provide a contemporary summary of statistical and non-statistical meta-analytic procedures that have relevance to the type of experimental designs often used by sport scientists when examining differences/change in dependent measure(s) as a result of one or more independent manipulation(s). Using worked examples from studies on observational learning in the motor behaviour literature, we adopt a random effects model and give a detailed explanation of the statistical procedures for the three types of raw score difference-based analyses applicable to between-participant, within-participant, and mixed-participant designs. Major merits and concerns associated with these quantitative procedures are identified and agreed methods are reported for minimizing biased outcomes, such as those for dealing with multiple dependent measures from single studies, design variation across studies, different metrics (i.e. raw scores and difference scores), and variations in sample size. To complement the worked examples, we summarize the general considerations required when conducting and reporting a meta-analysis, including how to deal with publication bias, what information to present regarding the primary studies, and approaches for dealing with outliers. By bringing together these statistical and non-statistical meta-analytic procedures, we provide the tools required to clarify understanding of key concepts and principles.